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1 .   BACKGROUND 

1.1   DEFINITIONS 

Given  the  binary  linear  integer  programming  problem 

Minimize  {cx|Ax<_b,  x=(0,l)}  (1 

where  A  is  an  m  by  n  matrix,  a  surrogate  constraint  for  prob- 
lem (1)  is  defined  as  follows: 


u(b-Ax)  >  0,    u  =  (u,,u0,...,u  ),    u  >  0.  (2) 

—  12m       — 


We  will  also  find  it  convenient  to  define  a  corresponding 
one-constraint  (knapsack)  surrogate  problem: 

Minimize  (cx|u(b-Ax)    0,  u  >_  0  ,  x  =  (0,1)}.        (3 

1.2   PROPERTIES 

Because  the  vector  u  0  implies  that  the  solution  set 
to  inequality  (2)  contains  the  solution  set  to  (1),  we  have 
the  following  properties: 

1.  If  x*  is  a  feasible  solution  to  (1)  it  is  also 
feasible  to  (2)  and  (3) . 

2.  If  the  surrogate  constraint  (2)  has  no  feasible 
solution,  then  neither  does  the  original  problem 
(1)  • 


1.3  USES 

Surrogate  constraints  are  used  in  conjunction  with  implicit 
enumeration  algorithms  (e.g.,  Balas)  in  several  ways.   Each 
vertex  in  an  enumeration  tree  represents  a  restriction  of 
problem  (1).   Problems  (1),  (2)  and  (3)  can  be  written  in 
explicit  terms  of  the  restriction  being  studied  by  substitution 
of  the  variables  assigned  values  by  the  restriction.   If  it 
can  be  shown  that  a  surrogate  constraint  corresponding  to  a 
vertex  has  no  feasible  solution  (completion) ,  then  the  vertex 
being  studied  can  be  fathomed  by  Property  (2) .   If  the  vertex 
cannot  be  fathomed,  Property  (1)  allows  the  surrogate  to  be 
appended  to  the  constraint  set  as  a  valid  constraint  to  be 
used  with  the  implicit  enumeration  tests  to  identify  promising 
variables  for  further  exploration.   Further,  if  the  solution 
to  Problem  (3)  corresponding  to  the  vertex  restriction  is 
known,  it  provides  an  upper  bound  on  the  original  problem  (1) 
if  it  is  feasible  for  (1)  or  a  lower  bound  on  the  vertex 
restriction  if  it  is  infeasible  for  (1) . 

1.4  STRENGTH  OF  SURROGATES 

In  order  to  choose  u  >  0,  Glover  [1965]  defined  surrogate 

1  2 

u  (b-Ax)  >  0  to  be  stronger  than  u  (b-Ax)  >  0  if 


1  . 

Min  {cx|u  (b-Ax)  >_  0 ,  x=  (0,1)}  >  Min  {cx|u  (b-Ax)  >  0,  x  =  (0,1 

for  u1,  u2  >  0.  (4) 


This  definition  states  that  if  the  corresponding  surrogate 


knapsack  problems  (3)  are  resolved,  the  surrogate  resulting 
in  the  more  restrictive  lower  bound  to  problem  (1)  is  stronger 
Essentially,  that  surrogate  eliminating  more  solutions  (as 
measured  by  the  objective  function)  is  the  stronger.   This  is 
intuitively  appealing  since  by  Property  (1)  the  surrogate 
cannot  eliminate  any  feasible  solutions  to  the  original  prob- 
lem.  Thus  by  this  definition  we  should  choose  u  as  follows: 


Max  Min      {cxju(b-Ax)  ^  0}  (5) 

u>0  x=(0,l) 


Unfortunately  (5)  is  difficult  to  solve  for  general  cases, 
although  Glover  [1965]  has  studied  the  two  constraint  case. 
An  approximation  to  (5)  can  be  made  by  relaxing  the  integer 
restriction  on  x,  i.e.,  choose  u  >  0  satisfying 


Max   Min   (cxju(b-Ax)  >_  0}  (6) 

u>0  0<x<l 


In  a  previous  report  the  strength  of  the  approximation  (6)  as 
measured  by  (4)  was  studied,  Giordano  [1982] .   In  this  report 
we  will  present  an  alternative  approximation  to  (5)  and  com- 
pare both  the  strength  and  speed  of  the  alternative  approxi- 
mation to  the  approximation  suggested  by  (6) . 

2.   THE  DUAL  MULTIPLIER  SURROGATE 

2.1   DEFINITION 

The  advantage  of  the  relaxation  to  (6)  is  that  it  can  be 
resolved  optimally  yielding  u   =  v   where  v   are  the  dual 


variables  to  the  linear  programming  (LP)  relaxation  of  (1) . 
Thus  at  any  given  vertex  restriction,  after  substituting  the 
variables  assigned  values,  the  LP  written  in  terms  of  the  re- 
maining free  variables  may  be  resolved  and  the  optimal  values 
of  the  dual  variables  used  as  weights  to  form  a  surrogate 
constraint.   A  surrogate  so  formed  is  called  a  dual  multiplier 
surrogate. 

2.2   PROPERTIES/USES 

As  with  all  surrogates,  if  it  can  be  shown  that  the  dual 
multiplier  surrogate  has  no  feasible  solution  then  the  vertex 
can  be  fathomed.   This  test  can  be  strengthened  by  requiring 
that  the  solution  to  the  surrogate  constraint  also  improve  the 
current  upper  bound  on  problem  (1)  ,  Geoff rion  [19  69]  .   After 
resolving  the  corresponding  LP  for  the  dual  variables,  the 
value  of  the  free  variables  may  be  used  to  solve  the  LP  relaxa- 
tion of  (3)  directly.   Note  in  this  case  when  solving  for  the 
dual  variables  we  are  solving  the  LP  relaxation  of  (1)  corres- 
ponding to  the  vertex  restriction.   If  the  values  of  the  free 
variables  are  integer,  problem  (1)  has  been  solved  for  that 
vertex  and  a  new  upper  bound  on  the  original  problem  has  been 
obtained.   If  the  values  are  fractional,  then  a  lower  bound 
for  the  vertex  is  obtained.   If  desired,  heuristics  may  be 
applied  to  the  fractional  values  to  identify  promising  varia- 
bles for  branching. 


2.3  COMPUTATIONAL  ADVANTAGES 

The  dual  multiplier  surrogate  has  been  widely  used  in 
conjunction  with  implicit  enumeration  algorithms  and  research 
has  been  conducted  on  frequency  of  use,  maximum  number  of 
constraints  to  carry  forward,  and  related  matters.   It  is 
interesting  to  note  the  effect  of  the  use  of  the  dual  multi- 
plier surrogate  on  the  problem  set  studied,  which  includes  18 
problems  with  up  to  50  variables  and  up  to  10  constraints. 
Seventeen  of  the  problems  required  a  total  of  552.36  CPU 
seconds  (CDC  6500)  using  a  Balas  Algorithm  with  heuristics. 
The  same  Balas  Algorithm  employing  a  dual  multiplier  surrogate 
generated  every  eight  iterations  and  carrying  a  maximum  of 
four  surrogates  forward  solved  the  17  problems  in  32.15  CPU 
seconds.   Problem  18,  consisting  of  50  variables  and  5  con- 
straints, had  not  been  solved  optimally  after  5631  CPU  seconds 
using  the  Balas  Algorithm  but  was  solved  optimally  in  6.4  7 
seconds  when  the  surrogate  was  added.   The  results  are  sum- 
marized in  Table  1,  which  is  found  at  the  end  of  this  report. 

2.4  OBSERVATIONS 

The  dual  multiplier  surrogate  has  been  a  very  significant 
contribution  to  implicit  enumeration.   Nevertheless,  there  are 
disadvantages  inherent  in  the  dual  multiplier  surrogate  when 
applied  to  large  problems.   A  linear  program  must  be  solved 
at  each  vertex  at  which  a  surrogate  constraint  is  to  be  formed. 
As  problems  with  larger  numbers  of  variables  are  considered, 
not  only  does  the  size  of  the  corresponding  LP's  increase, 
but  more  importantly,  the  number  of  vertices  grows  exponentially 


Since  one  of  the  primary  advantages  of  the  Balas  Algorithm  is 
that  it  is  additive  computationally,  the  necessity  of  solving 
the  LP's  should  be  investigated.   Note  that  the  necessity  to 
solve  the  LP's  makes  the  integer  programming  problem  more 
sensitive  to  the  number  of  constraints  than  is  otherwise  the 
case.   Ideally  one  would  like  to  build  a  surrogate  with  strength 
and  computational  advantages  comparable  to  the  dual  multiplier 
surrogate  but  requiring  less  computation  time. 


3.   AN  ALTERNATIVE  METHOD  FOR  CONSTRUCTING 
SURROGATE  CONSTRAINTS 


Definition  (4)  suggests  an  alternative  strategy  for  con- 
structing surrogates.   Given  an  initial  surrogate  a  stronger 
surrogate  can  be  constructed  by  making  the  optimal  solution 
to  the  current  surrogate  knapsack  problem  infeasible  for  the 
new  surrogate  constraint  while  continuing  to  eliminate  less 
optimal  solutions.   The  process  iterates  until  a  stronger 
surrogate  can  no  longer  be  constructed.   Such  an  iterative 
procedure  was  developed  when  the  strength  of  surrogates  was 
investigated,  Giordano  [1982].   In  the  referenced  report,  the 
initial  surrogate  was  the  dual  multiplier  surrogate  and  each 
surrogate  knapsack  problem  was  resolved  for  an  optimal  solu- 
tion using  a  Balas  Algorithm. 

To  develop  a  quick  heuristic  for  constructing  surrogates, 
two  major  problems  must  be  resolved.   First  an  alternative 
method  for  forming  the  initial  surrogate  must  be  developed 
since  solving  the  corresponding  LP  at  each  vertex  is 


computationally  costly.   Secondly,  an  alternative  process  for 
solving  the  surrogate  knapsack  problems  must  be  incorporated 
to  avoid  the  computation  time  involved  in  the  Balas  Algorithm 
We  will  review  the  procedure  for  iterating  to  a  best  surro- 
gate, investigate  alternative  methods  for  forming  the  initial 
surrogate,  present  an  approximation  technique  for  resolving 
the  surrogate  knapsack  problems,  and  compare  the  formation 
time  and  strength  of  the  heuristically  generated  surrogate 
with  the  dual  multiplier  surrogate. 

4 .   ITERATING  TO  A  BEST  SURROGATE 


4.1   AN  ALGORITHM 

Let  us  define  for  the  current  surrogate: 

x  :   the  optimal  solution  to  the  current  surrogate 

knapsack  problem, 
u*:   the  weight  of  the  ith  constraint  in  the  current 


1 


surrogate 


s.:   the  slack  x   allows  in  the  ith  constraint. 

S*:   7  u*s*:   the  slack  x*  allows  in  the  current 
0    4   i  i 

surrogate. 

Similarly,  for  the  new  surrogate  let: 


u! :   the  weight  of  the  ith  constraint  in  the  new 

surrogate . 
SI:   V  u.'  s*:   the  slack  x*  would  allow  in  the  new 

o   r  i  i 

i 

surrogate. 


Let 


* 
u; 
1      1 


The  purpose  of  a.  is  to  increase  the  weight  of  the  constraints 
violated  by  x  and  9  is  a  parameter  to  insure  x*  becomes 
infeasible  for  the  new  surrogate.   Choosing 


* 

a .   =   s . 
l      l 


and  9   =   (S*/£  s*2)  +  .05 


a  new  surrogate  is  generated  and  combined  with  the  previous 
surrogate,  weighting  the  previous  surrogate  75%.   If  a  suc- 
cessive surrogate  fails  to  be  stronger  than  its  predecessor, 
£  =  .05  above  is  halved  and  che  process  repeated.   If  a 
surrogate  fails  to  improve  after  three  reductions,  the  proc- 
ess terminates  with  the  previous  surrogate  judged  'best'. 

4 .2   COMPARING  THE  STRENGTH  OF  SURROGATES 

The  strength  of  a  surrogate  is  measured  by  the  optimal 
solution  to  the  corresponding  knapsack  problem.  To  compare 
various  surrogates  the  following  measure  proved  convenient: 


percent  convergence   =   |  z  '  -  z"  | / j z '  -  z 


where: 


z~:   objective  function  value  of  the  optimal  solution 
to  problem  (1) . 


z':  objective  function  value  of  the  optimal  solution 
to  the  continuous  relaxation  to  (1) . 

z":  objective  function  value  of  the  optimal  solution 
to  (3)  for  the  surrogate  in  question. 

The  percent  convergence  heuristically  measures  the  number  of 
infeasible  solutions  between  the  continuous  and  integer 
optimum  solutions  to  (1)  that  a  particular  surrogate  effec- 
tively eliminates  and  is  an  indication  of  the  relative  strength 
of  two  surrogates. 

4.3   RESULTS 

Of  the  13  problems  considered,  the  solution  to  the  dual 
multiplier  surrogate  knapsack  problem  solves  the  original  prob- 
lem directly  in  7  cases.   In  the  remaining  11  cases,  it  is 
possible  to  build  a  stronger  surrogate  in  9  cases.   The  im- 
provement in  most  instances  is  substantial.   In  fact,  the 
best  surrogate  obtained  solves  an  additional  4  problems.   The 
results  are  summarized  in  Table  2,  which  is  located  at  the  end 
of  the  report. 

5.   THE  INITIAL  SURROGATE 

5.1   METHODS  TESTED 

Three  alternative  methods  for  forming  the  initial  surro- 
gate were  tested: 

1.  Simply  adding  the  original  constraints. 

2.  Averaging  the  original  constraints. 

3.  Weighting  each  of  the  original  constraints  according 
to  the  right  hand  side. 
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5.2  DISCUSSION 

The  advantage  of  Method  1  is  that  it  is  a  simple  way  of 
getting  started.   Method  2  attempts  to  prevent  the  initial 
surrogate  from  becoming  so  large  that  it  compromises  the 
weighting  scheme  developed  in  4.1  when  forming  subsequent 
surrogates.   Method  3  attempts  to  exploit  the  format  of  the 
problem  for  the  vertex  being  studied.   In  the  Balas  format, 
the  problem  is  better  than  optimal  with  all  variables  at  the 
zero  level.   Variables  are  raised  to  the  one  level  only  to 
cure  inf easibility .   For  violated  constraints,  the  current 
right  hand  side  represents  the  inf easibility  which  must  be 
"cured".   Using  Method  3,  the  initial  surrogate  is  formed 
weighting  each  violated  constraint  according  to  its  relative 
inf easibility .   Various  normalization  schemes  were  tested  to 
insure  a  unit  of  slack  in  each  constraint  means  approximately 
the  same  thing. 

5.3  RESULTS 

For  the  problem  set  tested,  Method  2  generally  produces 
the  best  results.   Although  the  initial  surrogate  is  normally 
not  as  strong  as  the  dual  multiplier  surrogate,  the  process 
quickly  converges  to  a  "best"  surrogate.   Since  the  initial 
surrogate  is  not  as  strong  as  the  dual  multiplier  surrogate, 
some  loss  of  strength  is  experienced.   The  best  surrogate 
using  the  dual  multiplier  surrogate  as  the  initial  surrogate 
is  equaled  in  12  of  the  18  problems  tested.   More  importantly, 
the  best  surrogate  generated  using  Method  2  above  equals  or 
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improves  the  strength  of  the  dual  multiplier  surrogate  in  15 
of  the  18  cases.   The  results  are  summarized  in  Table  2. 

6 .   SOLVING  THE  SURROGATE  KNAPSACK  PROBLEMS 

6 .1   AN  APPROXIMATE  ALGORITHM 

Consider  the  knapsack  problem: 


Minimize  {7  (-ex.)  IT  a.x.  <  b,  x.  =  0  or  1}  . 

j     3    3       fj  3    3    ~      3 


By  preassigning  values  for  x.  where  c.  and  a.  differ  in  sign 

and  substituting  x.  =  1-x!  for  the  remaining  variables  where 

c.  and  a.  are  both  negative,  the  above  problem  can  be  reduced 

to  a  form  with  all  c.,  a.,  and  b  positive.   Arrange  the  indices 

of  x.  such  that  c,/a,  >  c~/a0  >  . . .  >  c  /a  .   We  will  refer 
3  11—22—     —  nn 

to  c./a.  as  the  knapsack  ratio  for  x..   Find  p  the  least 

integer  (0    p   n)  such  that     a.    b.   Beginning  with 

j<p   ] 
r  =  p  increment  r  by  unit  steps  to  n,  adding  each  a   to 
p-1  r 

1      a,  if  and  only  if  the  resultant  cumulative  sum  is  less  than 
k=l   K 
or  equal  to  b.   Then  the  approximate  solution  is  given  by 


.    (" 


if  j  <  p  or  if  a.  is  added  to  the  summation 


x . 


0,   otherwise. 


This  algorithm  arranges  the  variables  in  such  a  manner  that  the 
more  attractive  variables  are  elevated  to  level  one  before 
inf easibility  occurs,  Taha  [1975]. 
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6.2   INCORPORATING  THE  APPROXIMATE  KNAPSACK  ALGORITHM 

Since,  in  the  algorithm  presented  in  4.1,  it  is  only 
necessary  that  a  subsequent  surrogate  be  relatively  stronger 
than  a  previous  surrogate,  approximate  measurements  of  their 
strengths  should  be  sufficient.   When  the  approximate  algor- 
ithm is  incorporated,  the  desired  speed  is  obtained,  some 
loss  in  strength  in  the  'best'  surrogate  is  experienced, 
fewer  iterations  are  required  to  converge,  and  the  probability 
of  finding  a  feasible  solution  to  the  original  problem  in- 
creases.  Part  of  the  loss  in  strength  of  the  best  surrogate 
is  due  to  terminating  the  process  when  a  feasible  solution 
to  the  original  problem   is  found.   For  example,  in  Problem 
4,  the  approximate  solution  to  the  best  surrogate  constructed 
solves  the  original  problem. 

Since  the  approximate  solution  to  a  surrogate  constraint 
is  greater  (minimization)  than  the  exact  solution,  such  a 
solution  poses  a  greater  restriction  on  the  subsequent  surro- 
gate.  This  reduces  the  number  of  iterations  required  to  obtain 
a  best  surrogate  and  reduces  the  probability  of  building 
stronger  surrogates  in  the  vicinity  of  the  best  surrogate, 
since  no  surrogate  can  be  constructed  'between'  the  current 
surrogate  and  the  approximate  solution  to  the  current  surrogate 
The  fact  that  the  approximate  solution  is  greater  than  the 
exact  solution  to  the  surrogate  also  explains  the  increase  in 
the  number  of  feasible  solutions  to  the  original  problem  found. 
The  computational  advantage  of  using  these  feasible  solutions 
in  the  Balas  Algorithm  will  be  subsequently  discussed. 
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6.3   RESULTS 

In  the  18  problems  tested,  the  best  surrogate  obtained 
equals  the  dual  multiplier  surrogate  in  10  cases,  is  stronger 
in  4  cases,  and  weaker  in  4  cases.   Thus  the  two  surrogates 
are  roughly  equivalent  in  strength.   However,  the  heuristically 
determined  best  surrogate  is  formed  in  a  total  of  .89  seconds 
for  the  problem  set  compared  with  9.19  seconds  for  the  dual 
multiplier  surrogate.   The  results  are  summarized  in  Table  2. 

7.   ANALYSIS  OF  ANCILLARY  INFORMATION 

7.1   TERMINATION  OF  ALGORITHM 

The  algorithm  presented  in  4.1,  modified  to  incorporate 
Method  2  for  generating  an  initial  surrogate  and  the  approxi- 
mate technique  for  resolving  the  surrogate  knapsack  problems, 
terminates  under  the  following  conditions: 

1.  Stronger  surrogates  can  no  longer  be  constructed. 

2.  A  feasible  solution  to  the  original  problem  has 
been  found. 

Upon  termination,  one  always  has  a  solution  which  is  a  lower 
bound  for  the  vertex.   The  best  surrogate  formed  is  a  weighted 
combination  of  the  original  constraints  which  more  heavily 
weights  those  constraints  which,  in  some  sense,  are  critical. 
The  final  set  of  knapsack  ratios  thus  represents  the  attrac- 
tiveness of  the  variables  with  respect  to  the  critical  con- 
straints.  One  can  take  advantage  of  this  situation  to  attempt 
to  find  feasible  solutions  to  the  original  problem. 
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7.2  FINDING  FEASIBLE  SOLUTIONS 

When  the  algorithm  terminates  due  to  condition  1,  one 
knows  which  constraints  were  violated  by  the  solution  to  the 
best  surrogate.   Depending  on  the  format  (e.g.,  Balas)  and 
type  of  problem  considered  (e.g.,  set  covering),  one  may  be 
able  to  raise  additional  variables  to  the  one  level  in  order 
to  satisfy  the  violated  constraints.   The  last  set  of  knapsack 
ratios  can  be  used  to  determine  the  order  of  raising  additional 
variables  to  the  one  level.   For  the  18  problems  studied,  it 
is  possible  to  quickly  find  a  feasible  solution  to  the  original 
problem.   To  get  an  indication  of  the  effect  of  a  feasible 
solution  on  the  total  computation  time,  the  algorithm  employing 
a  dual  multiplier  surrogate  every  eight  iterations  carrying 
forward  a  maximum  of  four  surrogates  (2.3)  was  again  used. 
The  only  difference  was  than  an  initial  solution  to  use  as  a 
bound  was  provided.   The  18  problems  tested  requires  38.62 
seconds  to  resolve  without  the  bounds  and  19.97  seconds  with 
the  bounds.   A  total  of  .26  seconds  of  additional  time  is  re- 
quired to  find  feasible  solutions  for  those  problems  in  which 
a  feasible  solution  is  not  determined  while  iterating  to  a 
best  surrogate.   The  results  are  summarized  in  Table  1. 

7.3  HISTORY  OF  VARIABLES  AND  CONSTRAINTS 

When  iterating  to  a  best  surrogate,  one  may  use  index 
sets  to  record  which  variables  are  at  the  one  level  in  the 
solutions  to  the  various  surrogates.   This  information  can  be 
used  to  attempt  to  find  a  feasible  solution  or  for  developing 
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a  heuristic  for  branching  in  the  Balas  Algorithm.   If  addi- 
tional information  on  the  variables  is  desired,  the  exact 
solution  to  the  LP  relaxation  of  (3)  is  immediately  available 
once  the  knapsack  ratios  have  been  determined  [Dantzig,  1957], 

Similarly,  one  can  use  an  index  set  to  record  which  of 
the  original  constraints  are  violated  while  iterating  to  a 
best  surrogate.   This  information  can  be  used  advantageously 
in  the  Balas  Algorithm. 

8.   CONCLUSIONS 

Using  the  heuristic  procedure  developed  in  this  paper  it 
is  possible  to  generate  surrogates  of  strength  comparable  to 
the  dual  multiplier  surrogate  in  less  than  10%  of  the  time 
required  to  form  the  dual  multiplier  surrogate.   Because  of 
the  way  the  surrogates  are  formed,  one  would  expect  that  the 
time  required  to  converge  to  a  best  surrogate  would  behave 
well  as  the  size  of  the  problem  increases.   The  results  of 
the  experimentation  conducted  suggest  the  following  research: 

1.  Develop  an  algorithm  which  employs  the  heuristically 
generated  surrogate  in  a  manner  analogous  to  the 
dual  multiplier  surrogate. 

2.  Develop  heuristics  for  exploiting  the  ancillary 
information  developed  when  iterating  to  a  best 
surrogate. 

3.  Adapt  the  procedure  to  the  general  integer  problem. 
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TABLE  1 

EFFECT  OF  A  DUAL  MULTIPLIER  SURROGATE  AND 
AN  INITIAL  STARTING  SOLUTION  ON  THE 
SOLUTION  TIMES  OF  A  BALAS  ALGORITHM 
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TABLE  2 

COMPARISON  OF  THE  DUAL  MULTIPLIER 
AND  HEURISTIC  SURROGATES 
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